A Signal-processing Framework for Forward and Inverse Rendering a Dissertation Submitted to the Department of Computer Science and the Committee on Graduate Studies of Stanford University in Partial Fulfillment of the Requirements for the Degree of Doctor of Philosophy

The study of the computational aspects of reflection, and especially the interaction between reflection and illumination, is of fundamental importance in both computer graphics and vision. In computer graphics, the interaction between the incident illumination and the reflective properties of a surface is a basic building block in most rendering algorithms, i.e. methods that create synthetic computer-generated images. In computer vision, we often want to undo the effects of the reflection operator, i.e. to invert the interaction between the surface reflective properties and lighting. In other words, we often want to perform inverse rendering—the estimation of material and lighting properties from real photographs. Inverse rendering is also of increasing importance in graphics, where we wish to obtain accurate input illumination and reflectance models for (forward) rendering. This dissertation describes a new way of looking at reflection on a curved surface, as a special type of convolution of the incident illumination and the reflective properties of the surface (technically, the bi-directional reflectance distribution function or BRDF). The first part of the dissertation is devoted to a theoretical analysis of the reflection operator, leading for the first time to a formalization of these ideas, with the derivation of a convolution theorem in terms of the spherical harmonic coefficients of the lighting and BRDF. This allows us to introduce a signal-processing framework for reflection, wherein the incident lighting is the signal, the BRDF is the filter, and the reflected light is obtained by filtering the input illumination (signal) using the frequency response of the BRDF filter. The remainder of the dissertation describes applications of the signal-processing framework to forward and inverse rendering problems in computer graphics. First, we address the forward rendering problem, showing how our framework can be used for computing and displaying synthetic images in real-time with natural illumination and physically-based